Hi. In order to make things more concrete, let's make a direct comparison between three

instruments that we have gotten to know so far in order to deal with ill-posed innumerable

problems, which is the minimum norm solution, the TSVD reconstruction, and the Tikhonov regularization.

Okay, so the problem is F is equal to AU plus epsilon, and we always have the SVD U sigma

V transposed of A, and the minimum norm solution is given by A plus F, which is U times sigma

plus times V transposed, no sorry, it's the other way around. It's V times sigma plus

times U transposed F, and this can also be written as a sum from one, well, I have to

figure out the top index, but this will be one over sigma I, oh right, it's L, and one

over sigma I times U transposed, so U I transposed F times V I. Then there is the TSVD with,

let's say threshold alpha, then U TSVD alpha will be V sigma alpha plus U transposed F,

which is the sum I from one to R alpha, one over sigma I U I transposed F times V I, and

the Tikhonov with alpha will be, sorry, with lambda will be V sigma tilde plus lambda U

transposed F, and that is the sum over all indices actually. Is it N or is it M? It is

N. Sigma I divided by sigma I squared plus lambda U I transposed F times V I. Okay, in

order to understand those things better, let's plot the singular values first. So this is

I, this is sigma I, they're ordered, so, singular values doing something, and from some index

on they might be equal to zero and stay zero. So, one, two, and so on, and this is L, this

is the largest index such that the singular values are still positive, so on, and then

they're zero afterwards, so this is L plus one. How does TSVD work? We take the same

singular values, but we pick some threshold, let's say maybe this threshold with level

alpha, and then R alpha is this index here, such that it's the last index such that the

singular values are above this threshold, so this is sigma R alpha, and this is sigma

R alpha plus one. And this means that the reconstruction looks kind of like this, so

now we plot the one over sigma I values, because those are the coefficients that are used in

this decomposition here. So now, one over sigma I, of course, well, it goes to infinity,

so to speak, so the lower ones are quite small, and they grow, and they grow, and they grow,

and then we, well, I can't draw infinity, but let's just say, well, it's at plus infinity,

right, so they're stuck at plus infinity, so one over zero, we'll call this infinity.

We can also plot one over alpha, this is between the fourth and the fifth, so let's say here,

so this is one over alpha, and the indices are down here, and well, what happens now?

The minimum norm solution takes into account all singular values, which are strictly bigger

than zero. So those black dots are the coefficients that are used for the minimum norm solution,

so let's not make this black, let's use, I don't know, what's a good color for that,

maybe red, so minimum norm solution, well, we take this, we take this, we also take this,

take this, we take all the positive, so that the inverse is of the positive singular values,

and for everything else, we take zero, so take zero, zero, zero, and so on. What happens

for the TSVD? We pick this threshold one over alpha, and as soon as one over sigma i is

above one over alpha, we take zero instead, because the sum will then be empty. So, it's

probably not easy to make out, I'll draw it next to that. Okay, so here, here, here, we

don't take this, instead we take this and that, and then we take the same ones. I hope

that the visualization works. Let me maybe draw those lines in order to make that clearer.

Okay, and the red one goes up, goes up, goes up, goes up, goes up, goes up, and then only

drops down to zero. Now, the tricky one is, well, maybe purple, tricky one is the tick-on-off

one. And now let's do it like this, that we pick lambda equal to one over alpha. So here,

we have some threshold alpha. Now we pick lambda equal to one over alpha, which makes,

it will make sense, because this is the right dimensionality, so we can compare those two.

Now we can do this, let's try to visualize this function here. So this is, let's call

it f lambda of sigma i. So a quick sketch, f lambda of x is x divided by x squared plus

lambda. How does that look like? So this is x positive only, and this is f lambda of x.

If you plot that, it exits the origin with roughly, at emtotics, x divided by lambda.